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ABSTRACT 



Using a short form of the Student Assessment of Teaching and 
Learning (C. Ellett, J. Ruggutt, and D. Davis, 1999) and the Personal 
Learning Efficacy Measure (C. Ellett, J. Ruggutt, and D. Davis) in addition 
to a course evaluation form, this study investigated relationships between 
the college classroom environment variables of involvement, knowledge, 
professional skills, and higher order thinking skills and the dependent 
variables of academic self-efficacy and course evaluations. Participants were 
102 students enrolled at a southeastern university during the spring 2001 
semester. Multiple correlation analyses found that the predictor variables 
explained 32% of the variance in academic self-efficacy and 45% of the 
variance in course evaluations. All findings were statistically significant 
at the rejection criterion of p < 0.01. Similar research may facilitate 
understanding and improvement of college classroom environments . An appendix 
contains all three study instruments. (Contains 20 references.) (Author/SLD) 
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Abstract 

Using a short form of the Personal Learning Environment Measure and the Student 
Learning Efficacy Assessment, this study investigated relationships between the college 
classroom environment variables of involvement, knowledge, professional skills, and 
higher order thinking skills and the dependent variables of academic self-efficacy and 
course evaluations. Participants included 102 students enrolled at a southeastern 
university during the spring, 2001 semester. Multiple correlation found that the predictor 
variables explained 32% of the variance in academic self-efficacy and 45% of the 
variance in course evaluations. All findings were statistically significant at the rejection 
criteria of (p<.01). Similar research may facilitate better understanding and eventual 
improvement of college classroom environments. 
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REVIEW OF THE LITERATURE 
Introduction 

This study investigated the extent of relationships between the college classroom 
environment variables of involvement, knowledge, higher order thinking skills, and 
professional skill and the dependent variables of academic self-efficacy and course 
evaluations. The predictor variable of involvement measured the extent to which 
participants in the study perceived that they paid attention in class and understood course 
goals. The predictor variable of knowledge measured each of the participants’ 
perceptions of the extent to which facts, concepts, and principles were emphasized in the 
class. The predictor variable of higher order thinking skills (HOTS) measured the 
participants’ perceptions of the extent to which theories, problem solving, and creative 
thinking were emphasized in the class. The predictor variable of professional skills 
measured the extent to which writing skills, speaking skills, and career skills were 
emphasized in the class. 

The dependent variable of academic self-efficacy measured the extent to which 
participants thought that they could use their ability and effort to accomplish course- 
mandated learning objectives. Academic self-efficacy can be viewed as the participants’ 
appraisal of their capabilities to succeed in academic pursuits. The study’s second 
dependent variable of summative course evaluations measured the participants’ overall 
ratings of the courses in which they completed the study’s instruments. 
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Noting that very little research on the college classroom environment has been 
carried out, Ellett, Rugutt, and Davis ( 1 999) originated pioneering research into the 
relationships between college classroom environment variables and learning outcomes 
that included academic self-efficacy and course evaluations. These researchers found 
positive relationships of moderate strength between college classroom environment 
variables and the dependent variables of academic self-efficacy and course evaluations. 

The remainder of this literature review will discuss published literature concerning 
the college classroom environment, academic self-efficacy, and course evaluations. 

The College Classroom Environment 

Angelo (1990) discussed benefits of involving college faculty members in college 
classroom environment research. Describing classroom assessments as efforts to 
understand what students are learning and how they are learning it, Angelo reported that 
beneEts of college classroom assessment included favorable student responses to the 
assessment process and increased collaboration between faculty members about teaching 
and learning. Presenting instructor behavior as a central element in crafting college 
classroom environments that are appreciated by adult learners, Imtel (1991) contended 
that classroom instructors produce the best p)ossible classroom environments by involving 
the students in arranging the classroom climate and by being cognizant of their 
instructional roles in crafting the classroom environment. In the process of investigating 
the connections between college instructors’ self-perceptions and their perceptions of the 
classroom environments of their classes, DiekhofF (1992) found a statistically si gni ficant 
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(p< 01) relationship with a strength of (r=.491 between professors’ self-ratings and their 
ratings of their classes. The more positively they viewed themselves the more positively 
they viewed their classes. Imtel (1991) emphasized the influence of professors on 
shaping the college classroom environment and Diekhoff (1992) emphasized the positive 
relationship between professors’ perceptions of themselves and their perceptions of their 
classes. A synthesis of these two published authors’ views alludes to the benefits of 
professors developing positive self-perceptions as a means of encouraging themselves to 
develop positive perceptions of their classes. Considering the powerful influence that 
professors have on students’ perceptions of the college classroom environment it is 
advantageous for professors to enhance their self-perceptions in order to promote 
enhanced classroom environments that will be positively perceived by students. 

The Academic Self-Efficacv of College Students 
Invitational theory holds that the academic beliefs that students develop about 
themselves are influenced by the messages that they send and receive as participants in 
the classroom environment. As invitational theorists Purkey and Novak (1996) claim that 
college classroom instructors can deliberately transmit elevating and enabling 
communication that cause students to accomplish according to their potential. Positive 
invitations transmit the message that students are competent and dependable while 
negative invitations make students feel devalued. Concurring with invitational theorists, 
academic self-efificacy researchers contend that students’ academic self-efficacy is 
affected by the invitations (or the lack of invitations) that they receive as participants in 
the classroom environment (Bandura, 1997). As influential shapers of the classroom 
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environment, classroom instructors are in a position to powerfully influence academic 
self-beliefs that students develop about themselves (Pajares, 2001). 

Academic self-efFicacy (students’ appraisal of their capabilities to succeed 
academically) and the closely related construct of academic self-concept (students’ 
appraisal of their academic capabilities compared to other students) are important 
motivation-related variables that are positively associated with academic achievement 
and college success. Cockley (2000) found a statistically significant (p< 01) correlation 
with a strength of ( r=.46 ) between African-American college students’ academic self- 
concept and their grade point averages. House (1995) found positive relationships 
between college students’ academic self-beliefs and their grades in college mathematics 
courses. Watkins (1990) described a meta-analysis of educational research literature that 
found an average correlation of (t=A2) between academic self-concept and academic 
achievement. Gerardi (1990) found that academic self-concept was a strong predictor of 
the academic success of minority and low-income engineering majors. Presenting 
academic self-efficacy as a key academic self-belief, McCombs (1996) described 
having a positive sense of academic self-efficacy as a very favorable mental state that 
encourages students’ determination and perseverance to learn. 

Course Evaluations Completed bv College Students 

In the process of using the predictor variable of effective university teaching to 
explain 69% of the variance in the dependent variable of students’ perceptions of their 
learning, Sheehan and DuPrey (1999) articulated effective university teaching as 
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consisting of students’ perceptions of the extent to which the class was informative, 
challenging, and well organized. Although student evaluations of college courses have 
been increasingly used to evaluate faculty members since the late 1970s, their 
effectiveness as measurements of teaching effectiveness is attenuated by confounding 
variables such as: course difficulty and the professors’ grading practices, the professors’ 
personality and cultural background, the professors’ political position, and classroom 
chemistry (Nast,1999). Best and Addison (2000) found a statistically significant (p<.01) 
between the predictor variable of students’ perceptions of instructor warmth and course 
evaluations. These authors found that college students give higher evaluations to 
professors perceived as being warm and friendly than they give to professors perceived as 
being cold and distant. 

Method 

Participants 

The participants in this study included 102 students in the southeastern United States. 
Students in seven randomly selected classes including two sophomore-level classes, three 
junior-level classes, and two graduate-level classes completed this study’s instruments. 
Fifty-nine of the participants were females and forty-three were males. Sixty-two of the 
participants were Caucasian and forty were African-American and the participants’ 
socioeconomic backgrounds ranged from lower middle class to upper middle class. 
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Materials 

The first instrument used in this study was a researcher-developed short form version 
of the Student Assessment of Teaching and Learning (SATL) (Ellett; Ruggutt, and Davis, 
1 999). The first section of the short form of the SATL measured students’ perceptions of 
classroom involvement. The first section of the SATL has a split-half reliability 
coefficient of r=.67 . The second section measured the extent to which they gained 
knowledge fi-om the course by learning factual information, developing concepts, and 
applying rules. The knowledge section had a split-half reliability coefficient of r=.56 . 

The third section measured the extent to which students perceived that they gained higher 
order thinking skills fi"om the course by applying theories and by using problem solving 
and critical thinking. The higher order thinking skills section had a split-half reliability 
coefficient of r=.S7 . The fourth section measured the extent to which students perceived 
that they gained professional skills fi"om the course by developing job-related skills and 
wnting and speaking skills. The fourth section had a split-half reliability coefficient of 
.83. Ellett, Rugutt, and Davis (1999) have established the validity of the former items as 
being adequate for measurement purposes. 

The study’s second instrument was the Personal Learning Efficacy Measurement 
(PLEM) that measured the students’ academic self-efficacy by asking them to respond to 
six Likert-type items that measured the extent to which they exerted effort in the course 
to enhance their learning and the extent to which they thought their efforts could 
accomplish course objectives. Ellett, Rugutt, and Davis established the reliability of the 




9 



Learning Perceptions 9 



PLEM at r=.92 and these researchers also documented the validity of this instrument. 

The study’s third instrument measured students’ summative course evaluations by asking 
them to rate the quality of teaching in the course, the contribution of the course to their 
learning, and their overall rating of the course. 

The three instruments used in the study are in Appendix A. 

Design and Procedure 

The participants completed the study’s three instruments during the second week of 
April, 2001 in their regularly scheduled classes. The participants were told their rights 
and options according to research oversight protections at the sponsoring institution. 

They were told that their participation was voluntary and to feel free to take the option of 
not filling out the instruments. They were told that their responses to the instruments 
were anonymous. The classroom teacher left the room briefly while the researcher got the 
participants to complete the instruments. Total administration time was about ten minutes 
for each class. 

After all of the study’s participants had completed the instruments, the researcher 
scored the instruments and entered the data into the Minitab statistical software system. 
Null Hypotheses 

The first null hypothesis predicted no statistically significant (p<.01) relationship 
between the predictor variables of students’ perceptions of involvement, knowledge, 
higher order thinking skills, and professional skills and the dependent variable of 
academic self-efficacy as measured by the PLEM. The first null hypothesis was tested 
using multiple correlation and a correlation matrix was also run. The predictor variable of 
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involvement measured the extent to which each participant perceived that he/she paid 



attention in class and understood course goals. The predictor variable of knowledge 
measured each participants’ perceptions of the extent to which facts, principles, and 
knowledge were emphasized in the class. The predictor variable of higher order thinking 
skills (HOTS) measured each participants’ perception of the extent to which theories, 
problem solving, and creative thinking were emphasized in the class. The predictor 
variable of professional skills measured the extent to which writing and speaking skills 
and career skills were emphasized in the class. The study’s first dependent variable of 
academic self-efficacy measured the extent to which participants thought they could use 
their ability and effort to accomplish course objectives. The first null hypothesis was 
rejected and predictor variables explained 32% of the variance in academic self-efficacy. 

The study’s second null hypothesis predicted no statistically significant relationship 
(p<.01) between the predictor variables of involvement, knowledge, HOTS, and 
professional skills and the dependent variable of summative course evaluations. 
Summative course evaluations measured the participants’ ratings of the course. The 
study’s second null hypothesis was tested using multiple correlation and a correlation 
matrix was also run. Table 1 displays the results of testing the study’s first null 
hypothesis and Table 2 displays the results of testing the study’s second null hypothesis. 
The second null hypothesis was rejected and predictor variables explained 45% of the 
variance in course evaluations. 
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Table 1, 



The relationship between the predictor variables of involvement, knowledge, HOTS, and 
professional skill and the dependent variable of academic self-efficacy. 



PREDICTOR VARIABLES DEPENDENT VARIABLE 



Involvement (i=.55I * 



Knowledge fi=.35I * 



Academic Self-Efficacy 
R Square = 32 %* 



HOTS fr=.37I * 



Professional Skill ( i=.36I * 



^denotes a statistically significant (p<.01) relationship 
n=102 
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Table 2. 



The relationship between the predictor variables of involvement, knowledge, HOTS, and 
professional skill and the dependent variable of summative course evaluations. 



PREDICTOR VARIABLES 



DEPENDENT VARIABLE 



Involvement f r=.49I * 



Knowledge fr= 531 * 

Summative Course 
Evaluations 
R Square = 45%* 



HOTS Ir=.60l * 



Professional Skill fr= 61 1* 



\ 



* denotes a statistically significant (p<.01) relationship 
n= 102 
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Results of Testing Null Hypotheses 

The rejection of the first null hypothesis provided evidence of a statistically 
significant (p<.01) relationship between the predictor variables and the dependent 
variable of academic self-efficacy. The predictor variables, taken collectively, explained 
32% of the variance in the dependent variable of academic self-efficacy according to a 
multiple correlation test. A correlation matrix, which measured the extent of linear 
relationship between each of the predictor variables taken individually and the dependent 
variable indicated that students’ perceptions of classroom involvement had the strongest 
relationship ( r=.551 with academic self-efficacy. 

The rejection of the second null hypothesis provided evidence of a statistically 
significant (p<.01) relationship between the predictor variables and the dependent 
variable of course evaluations. The predictor variables, taken collectively, explained 45% 
of the variance in the dependent variable of course evaluations according to a multiple 
correlation test. A correlation matrix, which measured the extent of linear relationship 
between each of the predictor variables taken individually and the dependent variable 
indicated that professional skill (r=.61) and HOTS (r=.60) had the strongest relationships 
with course evaluations. 

Discussion 

The results of this study provide partial corroboration for the results of a pioneering 
college classroom environment study presented by Doctors Ellett, Rugutt, and Davis in 
1999. Doctors Ellett, Rugutt, and Davis found statistically significant positive 
relationships of moderate strength between college classroom environment variables 
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and academic self-efficacy and course evaluations. The college classroom environment 
and the effectiveness of college teaching and learning is a drastically under researched 
area that needs to be much more extensively researched. Correlation research is the ideal 
research framework for college classroom environment research because correlation 
research is logistically feasible, convenient, and highly replicable. Although correlation 
provides no evidence of predictor variables having direct causal impacts on the dependent 
variables, correlation does accurately measure the extent of mathematical association 
between variables that are important for enhancing the effectiveness of college teaching 
and learning. When replications of correlation studies reveal regularities such as 
consistently similar correlations between predictor variables and dependent variables 
progress is being made toward establishing general laws that have far-reaching 
applicability. 

Elkind (1994) described postmodernism as an attempt to replace science and 
scientists’ search for regularities with a search for differences. Educationists need to 
avoid the anti-science mentality that only looks for differences while not looking for 
relationships and regularities. Granted, differences are important and there are important 
differences between each university, each professor, each class, and each student. 
However, correlation can efficiently and accurately measure the extent to which 
universities, professors, classes, and students are similar in respect to their scores on 
variables that are important to the improvement of college teaching and learning. For 
instance, very similar relationships between students’ perceptions of classroom 
involvement and academic self-concept have been found (Knight & Waxman, 1990; 
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Byer, 2000; Byer, in press). Evidence exists of a consistent correlation coefficient of 



approximately (r=,30) between the predictor variable of students’ perceptions of 
classroom involvement and academic self-concept in secondary social studies classes. A 
similar, but slightly stronger, relationship of (i=.55) was found between these two 
variables in the present study. Repeated and consistent findings of similar relationships 
between variables that are important to the effectiveness of teaching and learning provide 
a basis for establishing and developing general laws. For instance, environmental press 
theory holds that the learning environment exudes a press (or influence). A positive 
environmental press promotes learning by encouraging students with beneficial 
advantages while a negative environmental press discourages learning by discouraging 
students with detrimental disadvantages (Murray, 1938). Correlation research into the 
college classroom environment adds to the knowledge base that is usable for establishing 
general laws and general theories that provide classroom instructors with general . 
guidance focused on improving teaching effectiveness. Differences still matter. Future 
research along the lines presented in this paper needs to examine differences in 
participants’ scores on variables according to grade classification and gender. 

Subsequent research also should investigate the extent to which instructors can 
intentionally improve students’ perceptions of key aspects of the college classroom 
environment. Subsequent research also should investigate the extent to which enhanced 
students’ perceptions of the college classroom environment are related to improvements 
in college learning outcomes including academic self-efficacy, course evaluations, and 



academic achievement. 
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APPENDIX A 



Student Assessment of Teaching and Learning (Short Form) 



PART ONE: (The INVOLVEMENT subscale) 

Directions: Please circle the number on the scale at the right which best reflects 
your feelings. 

Almost Seldom Some- Often Almost 
Never times Always 



1 . I know what I am trying to accomplish 1 
in this class. 

2. I pay attention in this class. 1 

3. I try to understand the work in this class. 1 



2 3 4 5 

2 3 4 5 

2 3 4 5 



Directions for parts two, three, and four 

Use the four-point scale below to evaluate the degree to which each type of learning is 

emphasized in this course. (DO NOT rate how much you have learned Only the 

amount of emphasis given to each type of learning. 

For parts two, three, and four rate the emphasis placed on each type of learning listed 
below by circling a number for each type of learning. 



1 = No emphasis 

2 = Some emphasis 

3 = Much emphasis 

4 = Very much emphasis 

PART TWO (The KNOWLEDGE subscale) 



1 . Learning factual information 12 3 4 

2. Developing concepts 12 3 4 

3. Understanding and applying principles and rules 12 3 4 
PART THREE (The Higher Order Thinking Skills (HOTS) subscale) 

1 . Understanding and applying theories 12 3 4 

2. Critical analysis and/or problem solving 12 3 4 
3 . Creative thinking 12 3 4 

PART FOUR (The Professional Skills subscale) 

1. Developing knowledge of self and others 12 3 4 

2. Developing professional, career, and job-related skills 12 3 4 

3. Developing written communication skills 12 3 4 

4. Developing oral communication skills 12 3 4 
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APPENDIX B 

The Personal Learning Efficacy Measurement 

Directions: Please respond to each of the following items by circling the number that best 
reflects your opinion about each question. 

1 . How much effort did you put forward in this course to enhance your own learning? 

Little or None Some A Large Amount 

1 2 3 4 5 



2. When there were difficult or uncertain obstacles to overcome in learning/ 
achieving in this course, how much effort and persistence did you put 
forward to enhance your own learning? 

Little or None Some A Large Amount 

1 2 3 4 5 

3. If you were repeatedly failing in this course, how much effort and persistence 
would you put forth to continue to enhance vour own learning ? 

Little or None Some A Large Amount 

1 2 3 4 5 

4. How much knowledge and/or ability do you think you have to accomplish vour 
learning obiectives in this course ? 

Little or None Some A Large Amount 

1 2 3 4 5 

5. How much personal responsibility do you think you have to accomplish vour 
learnin g objectives in this course ? 

Little or None Some A Large Amount 

1 2 3 4 5 

6. To what extent do you believe your efforts can accomplish the learning objective 
of this course? 

Little or None Some A Large Amount 

1 2 3 4 5 
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APPENDIX C 

Summative Course Evaluation 

Directions: Use the scale provided below and circle the appropriate number that best 
reflects the numerical grade you would give the course for each of the 
three items that follow. 

SCALE 

A = 90-100 
B = 80-89 
C = 70-79 
D = 60-69 
F = Below 60 

1 . How would you grade the quality of teaching in this course? 

100 90 80 70 60 50 40 30 20 10 0 

2. What was the contribution of this course to your personal learning? 

100 90 80 70 60 50 40 30 20 10 0 

3. How would you grade this course overall? 

100 90 80 70 60 50 40 30 20 10 0 

Demographic Characteristics 

Directions: Check the appropriate lines to identify your college, classification, and 
gender. 



College: 



Business 
Education 
Liberal Arts 
Natural Science/Math 



Classification: 

Freshman Gender: 

^Sophomore 

Junior Male 

Senior Female, 

Graduate Student 
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